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Re: Some thoughts on streaming 

Holger Grahn (holger@blaxxun.com) 
Sun, 14Jun 1998 20:48:34 +0100 

• Messages sorted by: [ date ][ thread ][ subject ][ author 1 

• Next message: David Chamberlin: "Re: Comments on streaming thoughts." 

• Previous message: Rex Brooks: "Re: Comments on streaming thoughts." 

• Maybe in reply to: Bernie Roehl: "Some thoughts on streaming" 

Thank you all for bringing up the "Some thoughts of streaming" 
thread, 

here are some collected ideas : 

To read as html http://www.snafu.de/-hg/stream.html 

Ideal solution MPEG 4 



MPEG 4 already describes solutions for the following items 

□binary encoding of a VRML scene graph 
□ dyjiamiciMod^ 
□streaming of the scene graph 
□streaming of audio, video and images 
□interleaving / multiplexing the different streams into 
one bit stream 

□VRML like authoring language 

Additional MPEG4 defines additional nodes for 

□facial animation 
□synthetic sound 
□2D VRML 
□Layouts / Composition 

By adopting/defining a pure MPEG4 VRML 97 subset, 
MPEG4^couldiberthed^ 

MPEG 4 sample : dyn amic sce ne ^graph:updat e with nodes * 

DEF GRP Group { 
children [ 

DEF TS TimeSensor { enabled FALSE } 

] 



AT 2000 { 

APPEND TO GRP.children 
Transform { 
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} 

} 

AT 3000 REPLACE TS.enabled BY TRUE 
Existing solutions or ideas 



blaxxun community server/CCpro VRML shared events. 

Basic VRMLr-field types can be routed from a clientzVRMLrsc.ene via the network to other clients. 
Server-side automated characters (Bots), or ser ver-side iscripts can stream-events4o~clients, 
or act dependir^on~eventsrcom^ 

DEF SharedZone BlaxxunZone { 
events [ 

DEF SharedRotation SharedEvent { name "newRotation" } 
DEF SharedTranslation SharedEvent { name "newTranslation" } 

] 

} 

DEF T Transform { 

} 

ROJtJTE SharedRotation. rotation_changed TO T.set_rotation 
ROUTE'SharedTranslation.translationchanged TO T.setjxanslation 

Sending 3 !*) the network can be done by routing:toJhe:set~type eyentfn of a SharedEvent. 

Bernies idea 



Bernies idea as described in http://ece.uwaterloo.ca/-broehl/streams/proposal.html 

For some type of applications it could be useful but I agree with 
David Chamberlain: Decoding must be done using a Java, there is no 
authoring time support for defining streamed data, the MovieTexture 
node gets over bloated. 

If encoding and decoding is propiertary to the application anyway , 

by adding a Raw Audio node, video & audio data could be streamed using 

a PixelTexture and a Raw Audio node from Java or the EAI. 

VRMEStfeahiinode 



Here is an idea for a simple, low-level VRML Stream interface 

Stream { i 
exposedFiell MFString url # HTTP / RTSP / UDP / TCP? 
exposedFiel JSFString contentType # the format of the data stream 
exposedField SFTime startTime # for starting the stream 



2 of 6 



1/28/03 2:03 PM 



vrml-streams archive for 1998: Re: Some thoughts on streaming 



http://www.web3d.orgAVorkingGroups/vrml-streams/hypermail/ 1 998/0 1 74.html 



exposedField SFTime stopTime # for stopping the stream 

exposedField SFBool loop TRUE # looping 

eventOut SFBool isActive # notification for start & stop of stream 

# delivery of stream data 

eventOut SFImage image_changed # contentType image/* movie/* 

eventOut MFInt32 audioFormat_changed # waveformat structure : channels, samples per second, 
bits per sample, encoding 
eventOut MFInt32 audioData_changed # raw audio PCM data packages 

# streaming a VRML with each top level node as unit 
eventOut SFNode node_changed 

# streaming basic field types 

eventOut SFFloat float_changed # contentType = "vrml/SFloat" 
eventOut MFFloat mffloat_changed # contentType = "vrml/MFFloat" 



} 

Together with a RawAudio node similar to David's Idea: 
AudioBufFer { 

exposedField MFInt32 format # waveformat structure : channels, samples per second, bits per 
sample, encoding 

eventOut MFInt32 data # raw audio data packages corresponding to encoding 

... / startTime / stopTime/ loop / lowWater ?? 
eventln SFFloat addSilence 
eventln SFTime flush 

} 

Usage scenario: 

Reading a image from some data file : 
DEF S Stream { 
url "myimage.jpg" 

} 

DEF T PixelTexture {} 

ROUTE S.image_changed TO T.set_image 

A Text ticker reading text strings from cgi-bin script 

DEF S Stream { 

url 7cgi-bin/text.cgi" 
contentType "vrml/MFString" 

} 

DEF T Text {} 

ROUTE S.string_changed TO T.set_strin 
Reading a movie 
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DEF S Stream { 
url "my movie, mpg" 

} 

DEF T PixelTexture {} 

ROUTE S.image_changed TO T.set_image 

DEF A AudioBuffer {} 

ROUTE S.audioFormat_changed TO A.set_format 
ROUTE S.audioData_changed TO A.set_data 

If the contentType is empty, the browser could derive it from the mime type of the URL. 
Supported formats would be raw-audio / wav, still and movie image formats. 
For contentType = vrml/a-vrml-field-type 

The data stream would simply field values coded in VRML utf8 ascii encoding : 
e.g. 

contentType vrml/MFFloat 

data : 

0.5 

[1.3 2.6] 
0.7 

for contentType vrml/* 

the field values could be preceded with the eventOut name e.g. 
float 0.5 

rotation 0 0 1 3.4 

node Material { diffuseColor 0.3 } 
MFVec3f[l 2 3,456] 

by adding a keyword "AT timeOffset" a relative event delivery time offset could be set. 

ATO 

float 0.5 

AT 3 // 3 seconds later 
float 0.7 

Once a VRML binary encoding standard is available, the first line of the data file would describe the 
encoding 

e.g. #VRML 2.0 MPEG4 

This idea is sounding relatively cheap to implement, seems to be flexible for many usage scenario, 
especially in combination with cgi-bin programs or direct TCP connections. 

Defining more complex streams with several interleaved data types, may need a sort of channel 
description and the channel information is encoded in the data stream 
A very rough sketch : 

Stream { I 
url "rtsp: ..." 1 
channels [ t 
DEFMoviel Channel { 
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target USE MovieTexturel 

} 

DEF Audio 1 Channel { 
target USE AudioClipl 

} 

DEF Audio2 Channel { 
target USE AudioClip2 

} 

DEF Motion 1 Channel { 

target USE MotionPacketl 

} 

] 

} 

PROTO MotionPacket { 

exposedField SFVec3f position 
exposeField SFRotation orientation 

} 

DEF MotionPacketl MotionPacket {} 

ROUTE MotionPacketl. position_changed TO myObject.set_translation 
ROUTE MotionPacketl .orientation_changed TO myObject.set_rotation 

The streanunode, recognizes some native^Brow^efliodes (MovieTexture, ImageTexture, AudioClip, 
Anchor) to automatically-route the networfedata to the nodes intemal-strea ming4nput-Bin? 
For other nodes the data format is described in a Proto interface. 

In MPEG4 there are objectand entit^stream desSiptors for this purpose. 



SMEL 



Defining sequences of animation's is currently quite authoring intensive in VRML. 
E.g. something like 
play animation 1 

at end of animation 1 start animation 2 with a 3 second delay 
parallel play moviel after that sound 1 

Several Scripts / TimeSensors must be controlled to get the effect. 
SMEL concepts could be expressed in VRML using new nodes : 

DEF My Animation Par { 
children [ 
Seq{ 

children [ 

DEF animationl Timer TimeSensor { cyclelnterval 20 } 
DEF waiterl TimeSensor { cyclelnterval 3 } 
DEF animation2Timer TimeSensor { } 

] 

} 

Seq{ 



I 
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children [ 
DEF moviel MovieTexture {} 
DEF soundl AudioClip {} 

] 

} 
] 

} 

ROUTE someSensor. start TO My Animation. set_startTime 

Seq and Par are controlling time dependent nodes, which have an set_startTime SFTime eventln, and a 
possibility to compute a duration, or monitoring an isActive FALSE eventOut 



TV or not TV ? 

I think its important to not understand under "VRML streaming" only a data downstream for yet another 
TV like presentation. 

Streaming should be possible in both directions, for multi-media communication or multi-user usage 
scenarios. 

-- Holger 
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